The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for protecting cognitive systems from gradient based attacks through the use of deceiving gradients.
Deep learning is part of a broader family of machine learning methods based on learning data representations as opposed to task-specific algorithms. Some representations are loosely based on interpretation of information processing and communication patterns in a biological nervous system, such as neural coding that attempts to define a relationship between various stimuli and associated neuronal responses in the brain. Research attempts to create efficient systems to learn these representations from large-scale, unlabeled data sets.
Deep learning architectures such as deep neural networks, deep belief networks, and recurrent neural networks, have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, and bioinformatics where they produced results comparable to, and in some cases superior to, human experts.
Neural network based deep learning is a class of machine learning algorithms that use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). Neural network based deep learning is based on the learning of multiple levels of features or representations of the data with higher level features being derived from lower level features to form a hierarchical representation. The composition of a layer of nonlinear processing units of the neural network used in a deep learning algorithm depends on the problem to be solved. Layers that have been used in deep learning include hidden layers of an artificial neural network and sets of complicated propositional formulas. They may also include latent variables organized layer-wise in deep generative models such as the nodes in deep belief networks and deep Boltzmann machines.